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ABSTRACT 

The conputer prograaing language APL is used to 
describe a ** learning** gaae, and the functions developed are 
generalized to extend to a class of rules. (Author/KH) 
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ABSTRACT 



APL is used to describe a "learning" game, generalized to a class of rules. 



Using a programaing language as a conceptual framework for describing 
topics has powerful implications for education — yet this approach is virtu- 
ally unexplored. The rewards for using a succinct and executable notation 
like APL are found in simple, clear expressions which often yield insights 
about the underlying nature of a topic. (See references [1] - [5]) APL is 
particularly well-suited for describing topics involving interactive pro- 
cesses, such as those encountered in gaming. 

In this paper , APL is used to describe a simple learning process in a 
computerized gaming context. A game provides a good environment for demon- 
strating learning by computer. Generally, games have well-dafined rules and 
objectives, narrow enough domains of discourse, and clear criteria for eval- 
uating performance. The meaning of "learning" is, of course, limited to the 
game environment and is usually defined in terms of improved performance on 
a specified task — such as making winning moves. 



The game chosen here to illustrate learning by computer is "LAST-ONE- 
LOSES" — a variant of the ancient intellectual game of NIM. LAST-ONE-LOSES 
is a two-person, zero-sum (win or lose) game with complete informiation which 
involves removing objects from an initial pile of objects. The rules are 
simple: One player moves first and may take 1 or 2 or 3 objects from the 
pile; the second player moves next, likewise taking 1, 2, or 3 objects. The 
players alternate moves in this fashion tintil there is only one object re- 
maining. Then, whoever must take the last one loses ! 

There exists an optimal strategy algorithm for LAST-ONE-LOSES; that is, 
from the start of the game, one of the players can always make moves which 
guarantee a win. Before continuing, the reader is invited to discover (and 
express!) the optimal strategy for this game. 

The prior question is: How does one formally represent this game — its 
interactive processes, simple s rategies, and mechanisms for learning? 



INTRODUCTION 



THE GAME OF "LAST-ONE-LOSES 



REPRhSSNTING THE GAMK 

To begin with, the basic interactive framework for two players to com- 
pete in this gaae of LAST-ONE-LOSES is enbodied in the program below. ^ 

V LASTONBLOSES 
ill PRINT EGGS'*- ?20 
122 MAKE:MOVE'*-n 
[3] PRINT EGGS'^-EGGS-MOVB 

-^-WIN IF EGGS^l 
C53 -^-LOSE IF EGGS<1 
£63 NEXTiMOVB-^-U 
C?] PRINT EGGS-^EGGS'MOVE 
C83 -^-LOSE IF EGGS'l 
C9] -t-WIN IF EGGS<1 
ClO] -^MAKE 

till irNt*I WON THIS GAME.* 
E123 H.0 

C133 LOSEi*YOU WON THIS GAME.* 
V 

V PRINT EGGS 
111 iEGGSro)pO 

V 

Here, the role of the computer is primarily that of mediator. It facilitates 

the playing of the game and announces the winner at the end of the game. 

For the computer to participate directly in the competition as one of the 

players, the following editing change is needed. 

^LASTONELOSESL 2 iMAKEi D^-MO VE-^-COMPUTERSi 

Also, a function must be developed to generate the COMPUTER 'i MOVE. 

Writing a function which makes legal moves at random is easy: 

7 MOVE-t-COMPVTBR Note that the minimum of EGGS and ?3 en- 

Cl3 MOVE*-EGGSl?Z sures that the resultant MOVE is never 

V larger than the number of EGGS available. 

However, as most children will quickly articulate, this function repre- 
sents an inferior strategy. It makes "dumb" moves in the most obvious situa- 
tions; it does not even remember its mistakes; and, clearly. It does not im- 
prove with any consistency from game to game. A function which learns as it 
plays would certainly be more interesting. 



Note that here the objects in the 
game are named EGGS. 



At start of the game, the number 
of EGGS is random. 



The messages on lines [11] and [13] 
are expressed from the point of view 
of the first player. 



PRINT displays the. appropriate number 
of goose-EGGS. 



■^Please observe that this program (as well as others to follow) is de- 
signed to be as simple as possible— with minimal concern for efficiency but 
high priority on readability . N.B. sub-function IF used in branching state- 
ments: 

V BRANCS<-LINE IF CONDITION 
C13 BRASCa<-COllDIfION/LINE 
V 



cn 

C2] 
[33 

[5] 



OQq 

One trivial kind of ieaming Involves avoiding losing sequences. Cf 
course » the COMPUTER could be prograauaed to store all sequences of moves for 
every game it played » to search through those sequences for one which occurred 
before, and — if it found a losing sequence — to be sure not to repeat it. 
This is a cuobersoBa learning scheme, although surely effective in the long 
run. 

Another approach to machine learning involves building in some 'intelli- 
gence* beforehand with a structure capable of adapting itself based on 'experi- 
ence*' (See reference [6] for details.) Such an "adaptive structure" provides 
a way of representing information about a game and utilizes rules for making 
changes in that infonaation. 

AN ADAPTIVE STRUCTURE 

In order to build an adaptive structure for the game of LAST-ONE-LOSES, 
first a global variable is created: CUPS'*^** 3pi3 (i call the variable CUPS 
because one can conceptualize this matrix as four paper cup«, each column con- 
taining the numbers 12 3.) 

CUPS 



1111 

2 2 2 2 

3 3 3 3 







CUP 1 



curs 



Next, a function must be designed to use the CUPS structure with a rule for 
making a move. The rule is in two steps: 

Step 1: Determine WHICH column of CUPS to use according to the current 
number of EGGS (as shown below): 
EGGS 123U56789 10 1112 13 1U 

WHICH 123412341 2 3 4 1 2 
Step 2: Then, PICK one of the numbers at random from that column (CUP). 
This rule can be embodied in a new definition of the COMPUTER function. 

i"':'''tH ''''' li\^":.7tiir^^^^^^ 



■*RAtJDUMB IF K/CUP^Q 
MO VE^-HQIS^-PICK ( CUP^Q ) / CVP 

BAN DUMB iMOVE-*-EGGSl ?3 



Where sub-function DETESMXIIE iMt 



of a CUP with all zeros. 

WHICH and >©VE record for later refer- 
ence the actual numbers used to gener- 
ate a MOVE. 

V WRICH^DETERMISE EGGS 
111 f/HICa^li-(N'H)\EGCS''l 
V 



ERIC 



and sub-function PICK is: 



V ONE-^-PICK MAflY 
C 13 Om*-MASn ToMAHY^^MARYl 
V 



00 :i * 



Now Che program which accomplishes the adaptive learning can be defined. 
The adapting process is simply: ''discard" the last KOV£ which led to a loss. 

V LBAEN This LEARN program replaces with a 0 
[1] CUPSl^Q^KBiiQil^'^^'O the nuober in the column of CUFS WHICH 

V represents the last (losing) MOVeT 

Finally » editing the aaln LAST-ONE-LOSES progran will allow the COMFllTER 
to LEARN after each game it loses :^ fLASTOSSLOSESll'*^ LSAR9 V 

LAST-ONE-LOSES now exhibits learning. 



LASTONELOSES 

00000000 0 0000 
1 

000000000000 
D: 

3 

000000000 

2 

0 0 0 0 0 0 0 
C: 

2 

0 0 0 0 0 
3 

0 0 
D: 

1 

0 

XOU wo It THIS GAME, 



After losing the game, the 
adaptive structure CUPS ip 
X \ 1 1 modified. Note that the last 

2 2 2 2 removed from CUP 1: 

0 3 3 3 



^To be more rigorous, i.e., to ensure that learning occurs only immedi 
ately after a game which COMPUTER lost but could have won, some additional 
editing changes are recommended. See Appendix for complete displays of all 
functions. 



00c 



u:;tosbloses 



0 
3 
0 
0: 



0 



0 



0 0 0 0 0 c 



0 0 0 



Again., CU?S has been adapt* ed to: 



0 
2 
0 
D: 



0 



1 

0 0 



CtlPS 



0 



1 



1111 

2 2 2 0 
0 3 3 3 



0 

lOU VOS TBIS GAME. 



Eventually y the adaptive structure becomes: 



CUPS 

0 10 0 
0 0 2 0 
0 0 0 3 

From that point on> COMPUTER will make the optimal MOVE; that is, it will 
win whenever a win is possible. 



Some insights can be drawn from the £inal state o£ the adaptive 
structure. Specifically, the pattern which emerged in CUPS reveals the 
modular nature of the game, and hence, provides a clue to an expression 
for the optimal strategy. The winning MOVE it ony time in the game when 
forcing a win sequence is possible) iu given by the expression ^\EGGS'l 
where EGGS is the number of objects r (Raining. Vlhen a win is not 
guaranteed (the other player has a f creed win), this expression yields 
0. VMtn HOfEwA^t es-^ll bft r^don and can be expressed as EGOS\.fZ 

Defined as a single functil>n» t':;ie optimal strategy is: 



THE OPTIMAL STRATEGY 



V MOVB-t-CPTIMAL 



CI] 
[2] 
[3] 



M0VE'**^\EGGS-1 
■^-0 Xr MOVE*0 
M0VE'*EGGSl?3 



TEACHING THE COMPUTER 



00/ 



With the existence of an optioal strategy function, one might think 
of using it to TEACH the COMPUTER, What better way to demonstrate 
machine learning than to have one conputer progran train anothcrl 

After supplanting the OPTIMAt function in LAST-ONE-LOSES, 
VIASTONBLOSBSZ 6 3 HBXTxMQ VE-^-OPTIMAL^ 
the process of leci.ning via an adaptive structure may be automated by the 
Tollowing TEACH prograai 

V TEACH HOWMAXI 
Cl] GAMES*-0 
12} PLAy-.LASTONELOSES 
C3] ■*'PLAy IF HOWMANX>GAMES'^GAMES-i'l 

V 

One Bight want to revise the LAST-OMB-LOSli iimgxaa to occlude unneceMary 

output and to record wlns/loMes, thualy: 



? LAST0HE10SES 

Zll EGGS-^ ?2Q 

C 2 ] MAKE : MO VE-^-COMPUTER 

C 3 ] EGGS-^-EGGS -MO VE 

Cu] -VJiV IF EGGS^l 

C5] -*^LOSE IF EGGS<1 

C 6 3 NEXT : MO VE*-OPTIMA L 

ni EGGS-^-EGGS'MOVE 

I 81 -t-LOSE IF EGGS-1 

191 -^-WIN IF EGGS<1 

[10 3 '*'MAKE 

nil WIHiRECORD^-RECORD^l 

C123 -►O 

C133 LOSE:RECORD'>-RECORD,'l 

Cl'*3 LEARN 



Note that the global variable 
RECORD must be initially specified 
elsewhere 



RECORD'- \Q 
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nisi COrV /^'-AHj^^LE 



Now, for those who wish to examin . patterns of learning in more 
detail — perhaps drawing analogies with biological learning — "learning 
curves'* can be readily produced. Based on values recorded during the 
TEACHlng» a two-dlmenalonal view of the COMPUTER'S learning can be 
graphed : 



iu 



o 

I 



TEACH 50 
GFAPS RECORD 



C 

□□ 

□□c 

□□CD 

□□□DC 

OODDOD 0 

DDDCDDCDD D C 
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r 

HP 
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DDCOPnnPDPCDPDPCnDDPPCDC 
DDDCDDDDCDCDDPOPDrPPOPPGO 

□oDCDonDnncDODDDonDPnpncnD 
□□□□□□□ocnpncnr lOPDCDDnnnoD 

DDCDnDDODDCnOOODnDGODDCCDPCP 
□GDCDOODDPnOCCCCnODDCDDDCPDnD 
CDDDGnDDDCnGDPnDPODDnnOPOCOCCP 

□DGccGDcnDGGcoGCDGoncpnrnnGnnPD 

C OGGCGGCDDOnDCCGCCDDGDnnPPGCCCCrr 



DDDDDDGGDQODGGGCCCCDQGDDDCDODCODDPGPDPGDrGGGCnCPDD 



The GBAPH function used ist 



V GRAPH RECORD 
ill NET'^*\RECORD 
C23 ADJUSTED'*'l-¥NET'l/HET 
C33 * U*tl-^i^\UAD JUSTED) ^'.iADJUSTEDl 

V 



N.B. TV^e •^N 

used OA It'netll 
is ntft fart 0^ 
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GENERALIZAJILITY 

Tbe APL functions developed so far can be generalized to extend to a 
clast.. of rules. Changes In the rules of LASX-ONE-LOS^XS can be handled sinply 
by substituting expressions In terms of a variable N -*> where the Integers iS 
are allowed In any give MOVE. Specifically, the functions OPTIMAL » COMPUTER, 
and DETERMINE need only substltude M for 3 and 1^1 for 4. (See Appendix for 
complete function displays.) Then, assuming that the appropriate global varl- 
ables are specified beforehand, 

one Is ready to play a generalized "learning" game. 

CONCIUSION 

This la but one topic — the topic of machine learning — ^whlch can be taught 
using a programming language as a conceptual framework. Many other topics are 
suitable for this pedagogical approach, not only topics from computer science 
and game theory, but also psychology, linguistics, statistics, social sciences, 
physical sciences, business, and ecology. Ifost fruitful are those topics which 
require explicit expression of Interactive processes or models* 

The challenge to educators, then. Is to Identify such topics and to lead 
students to better understandings through using A Progranmlng Language. 
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